2.1. Hierarchies and Metaphors in XML

The best full-scale introduction to the technologies that undergird TEI is found in Lou Buranrd's "Gentle Introduction to XML in Chapter 5 of the TEI:P5 Guidelines. " If you come from a History, Literature, or Philosophy department and have never worked with XML you may not find it quite so gentle, especially in its later chapters. This chapter addresses you and tries to lay out some basic facts about XML and TEI in language that stays as close to layman's language as possible. Examples are chosen from TEI Simple, but their point is to demonstrate general aspects of XML. Consult the "Tour of TEI Simple" for a detailed discussion of elements and their rules. The is some overlap in the discussions. If you are a novice, the repetition may be helpful. If you are an expert, you want to skip this section altogether.

A word of advice before proceeding any further. It is possible to use simple text editing software to work with XML, but if you value your time do not even think about doing it. Use an XML editor instead. An XML editor is a piece of software that "knows about" the general features of XML and the particular features of a distinct XML language such as TEI. It can save you countless hours by discovering (or even preventing) the many logical and typographical errors that are easy to make and hard to fix. Some XML editors are freeware. The software that will save you the most time is likely not to be free. There are several excellent XML editors. In the TEI world, oXygen is probably the most widely used.

XML stands for Extensible Markup Language. It is a set of rules for creating hierarchical data structures or Ordered Hierarchies of Content Objects (OHCO). You are familiar with such structures from the hard drive of your computer. It is a single container or "root" directory, which you can subdivide into an arbitrary number of subdirectories, each of which is "recursively" divisible into arbitrarily many subdirectories. Within the limits of the size of your hard drive each container is infinitely variable in size and adjusts to stuff that that you add or subtract.

The technical terms of XML make extensive use of tree and family metaphors. In XML parlance the root of a tree is envisaged as a single cell. In visualizations the tree is typically inverted so that the tree grows downwards from a root in the sky. Each branching of such a tree is called a node, and a node with no further branches is often called a leaf node. A good XML editor lets you present the outline or tree structure of your document in the form of a navigation panel.

As for the family metaphor, there is no sex in XML, and reproduction proceeds by parthenogenesis. A parent may have an indefinite number of child elements, but a child element has only one parent. It may have many ancestors, but at any level of the hierarchy there is only one of them. It is a defining feature of such hierarchical structures that for any node there is one and only path from the root to any node, or from any node to any other node. In XML the technical term for that path is XPath, which is both a name for such a path and a quite complicated computer language for getting from any "here" to any "there".

On your hard drive a container is known as a directory or folder. In an XML document, containers are known as elements. Container suggests volume and a three-dimensional structure, but the content of elements is text, which unfolds in a single temporal direction, envisaged from left to right in many, but by no means all, languages.

declare variable $collection := collection('/Users/martin/dropbox/speechdirectory');

for $title in $collection//filesDesc/titleStmt/title return $title

Why is the piece of code frame with a black border while its output is displayed in green field and with a green border? Anything that appears in green is a valid TEI XML fragment if it appears in the right context. Code framed in black makes no such claim.

As this example demonstrates, it takes a fair amount of prose to describe a very simple operation. Just about any individual operation with XML is quite simple, but there are a lot of them and they nest. Some of the most distinctive features and difficulties of XML are highlighted if you envisage the hierarchical structure as as one-dimensional stream. Imagine an XML document as a pair of parentheses with parentheses and nested parentheses within it. Nesting is a simple concept, but it is not easy for human minds or eyes to follow a nested structure beyond the third level. An interesting technical term in this regard is serialize. You don't need to know (I certainly don't) just how the data of an XML document are stored in the innards of a computer. But when they are produced by the machine for you in a manner that you can make sense of they are serialized. This serialization appears typically on the two dimensional space of a page, where the single text stream is broken into familiar lines. The typographical line has been a remarkably stable structure for millennia, with limits imposed by the capacities of the human eye. But it is helpful to remember that a line is a convenient construct of a longer discursive linear stream.

The parentheses that mark the boundaries of XML elements are more complicated things than round, square, or curly brackets. They are constructs enclosed in angle brackets, are known as tags, and have names. The opening tag of an element with the name "someElement" looks like <someElement>, while the closing tag adds a solidus after the left angle brackets:</someElement>. As with a directory on your computer, you can put as much stuff between those tags as your hard drive will hold. The following is a well-formed XML document

<someELement>Mary had a little lamb</someELement>

But so is

<someELement> In the beginning God created the heaven and the earth....The grace of our Lord Jesus Christ be with you all. Amen.</someELement>

with all the other words of the Bible in between.

2.2. Well-formed XML

The term well-formed has a very specific meaning in XML. An XML text must be well-formed, and it may be valid. For an XML document to be well-formed it only needs to meet the conditions that must be true of all XML documents. There are not many of them, but the two most important are that a) the document must be enclosed by a root element, and b) that the name of an element must be spelled exactly the same way in the opening and closing tag. Thus

<someElement>Mary had a little lamb</someelement>

is not a well-formed XML document, because XML is case-sensitive and the spelling of the closing tag "someelement" does not match the spelling of the opening tag "someElement." Neither is

<line>Mary had a little lamb</line> 
<line>Its fleece was white as snow</line>

But

<linegroup> 
<line>Mary had a little lamb</line> 
<line>Its fleece was white as snow</line> 
</linegroup>

is well-formed XML because the two <line> elements are enclosed by the root element <linegroup> and the opening and closing tag names are spelled the same way.

These simple examples of well-formed XML consist of element wrappers that enclose text. From a reader's perspective XML has slightly awkward ways of referring to ‘just text’. The software that processes an XML text is known as a parser, a term borrowed from traditional grammar. Just as you parse a sentence by determining where its different parts begin and end, so an XML parser goes through an XML file and inspects its parts. It looks for opening and closing tags of each element as well as the stuff enclosed by them, which consists either of other elements or of PCDATA or parsed character data, stuff that the XML Parser after examination has recognized as proper text.

Except for the left angle bracket (<) and the ampersand (&) any Unicode character can show up in PCDATA. Because the XML parser uses (<) to identify the beginning of an element tag, it cannot be used directly in plain text or PCDATA. Instead you use a character entity to represent them. A character entity is a periphrastic expression that is wrapped in an opening ampersand and a closing semicolon. Since there are no words that begin with the former and end with the latter they act as an unambiguous marker that their content refers to and retrieves something special, whether a single character or all of Hamlet. The character entity for the left angle bracket is (&lt;), where ‘lt’ stands for ‘less than’ to reflect the common use of that character in mathematical notation.

Because the ampersand is used as the opening symbol for a character entity, it also must be escaped and cannot be used directly. Its character entity is "ampersand + amp + semicolon" (&amp;). These character entities are two of five predefined entities in the XML specifications that you can use without formally declaring them. The other three are the apostrophe ('), the double quotation mark ("), and the "greater-than" symbol or right angle bracket (>). Of these, the apostrophe entity is useful if you want to distinguish between the uses of the same character as the genitive marker and a single quotation mark.

Within the range of TEI Simple you probably will not need to know any more about character entities, but you can find out more about them at XML Entity Definitions for Characters (2nd Edition).

2.3. Valid XML

The requirement that everything in a well-formed XML document must be enclosed by a root element does not take you very far. A well-formed directory structure on a hard drive can be, and often is, a complete mess. XML becomes interesting and useful only when it is valid. XML has rules for making rules that specify for any element

• what elements it may or must contain or be contained by
• how often it may appear in its parent elemtn
• in what order it may or must appear

A set of such rules is known as a schema. You can use these rules about rules to make up your own schema, but if you want to share your documents and make sure that others can use them you are better off using a schema that has been designed by others for general use in some domain, whether Chemistry, Mathematics, or whatever. TEI is such a schema, and its purpose is to facilitate the encoding of documents that fall under the category of Letters very broadly defined.

How does a machine know that a document is an XML document,whether with or without a schema? Any XML document must begin with one or more processing instructions that tell the machine what to do with it. The minimal processing instruction simply says ‘I am an XML document’ and does so with the line

<?xml version="1.0" encoding="UTF-8"?>

at the top of the file. The statement ‘I am a TEI Simple document’ is a more elaborate affair:

<?xml-model href="http://raw.githubusercontent.com/TEIC/TEI-Simple/master/teisimple.rng" type="application/xml" schematypens="http://relaxng.org/ns/structure/1.0"?> 
<?xml-model href="http://raw.githubusercontent.com/TEIC/TEI-Simple/master/teisimple.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>

You can encode quite successfully in TEI Simple by just cutting and pasting this code. If you work with a program like oXygen, you can choose a new document as a TEI Simple document, and it will open with all the required processing instructions. It is, however, useful to hava rough idea of what is going on here. A processing instruction comes in a <? ...?> wrapper, which answers to the machine's question ‘What shall I do here?’ In our example there are two instructions. The first one says

The second instruction, which is rather more cryptic, specifies that the rules for the <teiHeader> element differ from the rules for the <text> element.

A schema is a very technical document and can be expressed in various schema languages. In the TEI world Relax NG is the most popular schema language. You can mostly treat the TEI Simple schema (or any other schema) as a black box whose operations you trust. Until about a dozen years, working with XML texts was a painful exercise. XML texts are equally unforgiving when it comes to orthographic or logical errors. Both are very easy to make and very hard to find or fix in a document of any complexity. Modern editing software wil validate your document as you work on it. That is to say, it will read your document and check whether it complies with the rules declared in the schema. If it discovers an error, it will tell you about the what and where of that error right then and there. It is a good investment of your time to learn how to read such error messages.

Although working with XML is much less painful than it used to, you still have to understand the particular rules that are formulated in the arcane language of the schema. You also need to know something about the rule-making rules of XML in order to work with TEI documents. And it helps to remind yourself constantly of some basic facts about XML. An XML document is in principle human-readable. If the only surviving copies of Dante's Inferno, Cervantes' Don Quixote, or Shakespeare's Hamlet were XML versions they could be read by humans who know Italian, Spanish, or English. But an XML document is not made to be read by a human. It is made to be processed by a machine for some purpose, whether displaying it on a screen, printing it out in some formating, or retrieving bits of information from it. An XML document is to a considerable extent agnostic about the purposes to which it may be put. It is not a single purpose version of its source. In particular, it is not designed as a reader-friendly document, although you may generate a reader-friendly version from it.

2.4. XML as a declarative markup language

2.5. The Nature of XML rules

TEI is a specific markup language designed to encode texts that originated in a print or manuscript world. You can also use it to generate born digital documents, but you will understand its quirks more readily if you keep its original purpose in mind. The XML rules for specifying the TEI markup language have to do with

• what elements may contain or are contained by other elements
• the order of child elements within a particular parent element
• the number of child elements that a parent element may contain

The words may, must and may not are important technical terms that the machine can understand when it validates an encoding. There is also the ocasional should, but should is not a term the machine can make sense of. It involves conventions that it is helpful to observe, even though for one reason or another they cannot be expressed in terms of may, must or may not.

2.6. A simple example of the rules

The skeleton of a TEI document provides a good example for the ways in which the rule-making capabilities of XML are exercised. A TEI document MUST have one and only one root element, and it name is TEI. The element name TEI is a specific feature of the TEI schema. The requirement for a root element is a requirement of well-formed XML.

The TEI root element MUST have two child elements, <teiHeader> and <text>. There MUST be only one of each, and <teiHeader> MUST come before <text>. These are TEI specific rules, but they observe the rule-making rules of XML.

<teiHeader> and <text> relate to each other as a catalogue record to the item it catalogues. <teiHeader> consists of data about <text> or metadata.

Encoding the header is a very different thing from encoding the text, and information from the header is never included in a representation of the text. We will leave the header alone, except for noting briefly what MUST be in it. Here is an example of a minimal header:

The header MUST have a <fileDesc> element or file description, which answers at least the following three questions:

• What is my name? (<titleStmt>)
• Who published me? (<pubStmt>)
• Where do I come from? (<sourceDesc>)

The <fileDesc> element MAY have a lot of other child elements, but it MUST have <titleStmt>, <pubStmt>, <sourceDesc>, only one of each and in that order. This is a very simple example of XML rules codifying practices from some domain, here the world of published books. The normal case of a TEI document represents a file that has been digitized and published from a printed source. But this hypothetical example represents a digital file that has not been published and was born digital.

In this simple example, <titleStmt>,<pubStmt>,and <sourceDesc> do not have text content, but their child elements <p> and <title> do. Some elements can only contain other elements, some elements can contain mixed content, i.e. text nodes and child elements. Some elements typically contain only text nodes, but TEI Simple has no element that MUST contain only a text node.

So much for the header, at least for now. Let us turn to <text>, the container for encoding the actual document, including its title page if it has one. A book is a bibliographical unit, which may contain one or more work(s). The second quarto of Hamlet is a book with one work. The Shakespeare Folio of 1623 contains three dozen separate works. While the TEI can handle both situations, we will focus here on the normal case of ‘one book, one work’, but it is useful to acknowledge the fact that the relationship between book and work and be very complex and raises tricky encoding questions.

The <text> element MUST contain one and only one <body> element. It MAY contain a<front> or <back> element, or both of them, but only one of each. <front> MUST precede and <back> MUST follow <body>. This is another example of perfectly obvious and sensible rules from some domain being codified in the rigid rules of the XML schema. Since you are very unlikely to encode the sequence <back>, <front>, <body>, you will not benefit from the validator's telling you that this is not valid XML, but there are many other and less obvious cases where the validator is helpful.

The chapter or section level <div> element is one of the TEI workhorses. You MAY have as many of them as you like in your <front> <body> or <back> elements, and you MAY nest them as deeply as you like.

The paragraph level <p> element is the other workhorse of the TEI. Any element that MAY contain <p> elements MAY contain any number of them. Unlike <div> elements, <p> MAY contain plain text or PCDATA (and typically will consist mostly of it), but a <p> element MAY NOT nest within another <p> element. This is a perfectly sensible rule. The <p> element is based on the model of the paragraph. Paragraphs divide into sentences; you don't expect them to divide into sub-paragraphs. Or, if there is a document with paragraphs and sub-paragraphs, you are probably in the world of lists and list items. The TEI has elements for lists and their items, and a list item MAY have a <list> child element. But even though <p> MAY NOT be a child of <p>, it is possible that a valid child of <p> could have a valid child element <p>. Whether you SHOULD use such encoding is another question.

It is useful to think of most TEI elements as operating mainly at one of three levels:

• Text structure elements elements articulate the macrostructure of your document, e.g. <front>,<body>,<back>,<div>. With some very specialized exceptions, such elements MAY NOT contain PCDATA.
• Block elements, principally <p> and <l> contain stretches of plain text whose end is almost universally marked by a line break in print.
• Inline elements operate at the microstructure of discourse. Printed text will often use italics or some other device to highlight a word or phrase. It is often, but not always possible to determine the reason. In TEI the <hi> element is the all-purpose element to encode inline words or phrases that have been highlighted.

But wherever one draws a line, there are phenomena that want to cross that line. In TEI there are several inter elements that can occur within paragraphs (inline) or between paragraphs (block). The most common of these is the versatile and slippery <q> element. Paragraphs often end with a quotation in a pattern like the following:

The fact that the paragraph does not end with a period but with a colon might lead you to think of the quotation as part of the previous paragraph. It does, however, lead to a deeply nested structure in which <p> elements are the descendants of <p> elements. If you like #5 of the Zen of Python (‘flat is better than nested’) you might decide that the paragraph is terminated by the colon and that <q> should be the right sibling rather than child of <p>:

That is a very different case from a quotation of two lines of verse in the middle of a sentence, as in

In the latter case there is no ambiguity or choice, and there is nothing strained about <l> as a descendant of <p> In the former case, there is ambiguity, and you may want to choose the option that avoids strained nesting. With nested structures it is useful to follow the advice ‘as much as necessary but as little as possible’.

2.7. About Namespaces

Namespaces are a little like surnames, letting you distinguish between Miller, Henry and Smith, Henry. For instance, HTML and TEI both use elements with the names <p> and <div>. But these are "false friends" with very different meanings and operating rules. By convention a namespace takes the form of a URL. The full name of an element is known as its "qualified name" or QName, and in one conventional notation it is written by prefixing the element name with the namespace in curly braces Thus the namespace of TEI is http://www.tei-c.org/ns/1.0, and the full name of a TEI element such as <p> is {http://www.tei-c.org/ns/1.0}p.

Because QNames are very clumsy things to manipulate, shortcuts have been invented. If you map the namespace to a prefix, tei:p will stand for {http://www.tei-c.org/ns/1.0}p, five instead of thirty characters. The choice of the prefix is arbitrary, but the prefix and element name must be connected by a colon. Unless you manipulate an XML text with XSLT, xquery, or some other language, you don't have to worry much about namespaces. But it is useful to know about them. If you do have to use them remember that they are fussy and a common source of error.

2.8. About Attributes

Elements can have attributes. The term is borrowed from grammar, and attributes relate to elements pretty much as adjectives relate to nouns. An attribute consists of a key-value pair, such as type='act', where the term before the equal sign is the name of the attribute and the term after the equal sign is the value. The value must be in quotation marks,whether single or double, but smart quotation marks will raise an error message. Thus <div typ="act" n=3"> would be a way of specifying a particular <div> as containing "Act III" of some play.

In well-formed XML, there are rules about the names of attributes, but no rules about what attributes to use with what elements or what values to associate with the attributes. In valid XML, the rules for attributes are specified in the schema. There are rules about what attributes may be used with with what elements and what kinds of data may, must, or may not appear in attribute values. In the TEI schema some attributes are global and can be used in the same manner for any element. Other attributes are specific to a particular set of elements.

A few of the global attributes in the TEI schema belong to the xml namespace. The two that you are likely to encounter often are xml:id and xml:lang. The former specifies an ID for an element that is unique within the document. The latter specifies the language in which the document (or part of it) is written. The rules for these two attributes are not TEI specific but follow the general rules for XML. That is why instead of id and lang these attributes always appear with their XML prefix xml:id and xml:lang.

In the TEI schema the value of an xml:id attribute must be a proper XML Name: it must begin with an alphabetic character; it may include digits, periods, hyphens, and underscores, but not commas, or semicolons. Because the colon required to link a namespace prefix to an attribute name you should only use it for that purpose. TEI has its own rules for what values may, may not, or must appear in different attributes.

The two most widely used global attributes in TEI are n and rendition. The name of the former is short for "number," and numbering of one kind or another is its most common, but not its only use. There are hardly any restrictions on what you can use for its attribute values. You can even, while you are working with a document, embed comments, as long as you remember to remove them in time. If n="5" is a proper value, so is n="5 check this again later." But you will be better off using the formal commenting protocol of XML, wrapping any kind of comment in <!-- [my comment] -->

Other attributes values are much fussier. The <l> supports a part attributed with a fixed list of values Y, N, I, M, F for yes, no, initial, medial, and final. The value for the target attributes for the linking element <ref> may consist of one or more URL's separated by white space.

The type attribute, which is available for some but not all elements, is a very powerful device for making more granular distinctions among kinds of discourse, especially when it is combined with a controlled vocabulary. For instance, the TEI schema include a <lg> element for encoding any kind of grouping of lines of verse. A controlled vocabulary for tagging stanzaic and metrical patterns is a great device for enhancing the query potential of a corpus of poems.

2.9. Limitations of TEI as an XML Markup Language

As an XML markup language TEI is committed to two axioms:

• A document is an OHCO or ordered hierarchy of content objects
• The presentation of a document can be neatly separated from its structure

Both of these axioms are deeply problematical when it comes to texts, but they are true often enough to be useful. Many texts divide neatly into single hierarchical structures, but others do not, and rhetorical effects frequently rely on the reader's or listener's effortless management of what in TEI parlance is called overlapping or concurrent hierarchies. Take the last two lines of Hamlet's soliloquy:

The fair Ophelia! Nymph, in thy orisons

Be all my sins remembered.

The prosodic structure of two lines of blank verse is counterpointed by a syntactic structure in which a sentence begins in the middle of one line and spills over without a break into the next line. You might be tempted to encode this as follows

The second axiom, that content is entirely separable from presentation, will rub most literary critics the wrong way. If you are used to reading a book printed in Fraktur you may feel that printing it in Antiqua robs it of its spirit. In Vergil's Eclogues up or down things happen at oddly regular intervals. Vergil appears to have written these poems on page blocks with forty lines and knew when he was writing at the top of page (nice words about Maecenas) or at the bottom (bad things happening to bad guys). There are countless examples of playing with layout and typography, from George Herbert to concrete poetry.

If you make the physical layout of a text the organizing structure you can do that by using the <surface > element and its children. That is also beyond the scope of TEI Simple, but it is fully documented in Chapter 11 of the Guidelines

TEI is about text, and texts are written in natural languages. Natural Language Processing (NLP) is a very important part of computing. If you engage in it you will discover early on that human language is highly rule-bound, that you can write procedures that will allow a machine to capture many of these rules very accurately and at great speed, but there is always an algorithmically intractable residue. St. Jerome, the translator of the Bible into Latin, spoke of ‘ipsum... suum et, ut ita dicam, vernaculum linguae genus’ (the distinct and as it were vernacular nature of each language) and the 16th-century French poet Du Bellay argued that literal translation destroys the ‘je ne scay quoy’ of a language. Pope wrote of a ‘grace beyond the reach of art’. Text encoding is a way of enabling a machine to process texts in ways that help human readers "make sense" of them. The machine can take you quite far, but it will only take you so far. In an encoding project of any complexity there will always be some moments when you remember this line from Wallace Stevens' Connoisseurs of Chaos:

2.10. Should, Should Not, and Tag Abuse

Valid encoding is not necessarily good encoding, just as rigid compliance with the law does not always square with justice. Here we are in the realm of Aristotle's epieikia or equity as a principle that complements rigid compliance with a set of rules. The word lives in a space tenuously circumscribed by the probable, the reasonable, and the appropriate--very much like the space of the French twins vraisemblance and bienséanceWhat SHOULD you do when confronted with a problem to which the schema offers no obvious solution? Do as your neighbours do is good advice when you live in a good neighbourhood, and as a TEI encoder you happen to live in a very friendly and helpful neighbourhood. Sign up for the TEI listserv and ask for advice. It will come to you from just about any continent, very promptly at any hour of the day or night, and it is nearly always relevant. Following the TEI list and pondering its advice is the best way of getting a feel for the discretionary space surrounds all rulles, and it will keep you from committing tag abuse, the TEI community's term for encoding practices that are valid but make no sense or worse.

3.1. A Short Example

We begin with a short example, intended to show what happens when a passage of prose is typed into a computer by someone with little sense of the query potential of a digitally encoded text. In an ideal world, such output might be generated by a very accurate optical scanner. It attempts to be faithful to the appearance of the printed text, by retaining the original line breaks, by introducing blanks to represent the layout of the original headings and page breaks, and so forth. Where characters not available on the keyboard are needed (such as the accented letter a in faàl or the long dash), it attempts to mimic their appearance.

                                CHAPTER 38

READER, I married him. A quiet wedding we had: he and I, the par-
son and clerk, were alone present. When we got back from church, I
went into the kitchen of the manor-house, where Mary was cooking
the dinner, and John cleaning the knives, and I said --
  'Mary, I have been married to Mr Rochester this morning.' The
housekeeper and her husband were of that decent, phlegmatic
order of people, to whom one may at any time safely communicate a
remarkable piece of news without incurring the danger of having
one's ears pierced by some shrill ejaculation and subsequently stunned
by a torrent of wordy wonderment. Mary did look up, and she did
stare at me; the ladle with which she was basting a pair of chickens
roasting at the fire, did for some three minutes hang suspended in air,
and for the same space of time John's knives also had rest from the
polishing process; but Mary, bending again over the roast, said only --
   'Have you, miss? Well, for sure!'
   A short time after she pursued, 'I seed you go out with the master,
but I didn't know you were gone to church to be wed'; and she
basted away. John, when I turned to him, was grinning from ear to
ear.
   'I telled Mary how it would be,' he said: 'I knew what Mr Ed-
ward' (John was an old servant, and had known his master when he
was the cadet of the house, therefore he often gave him his Christian
name) -- 'I knew what Mr Edward would do; and I was certain he
would not wait long either: and he's done right, for aught I know. I
wish you joy, miss!' and he politely pulled his forelock.
   'Thank you, John. Mr Rochester told me to give you and Mary
this.'
   I put into his hand a five-pound note.  Without waiting to hear
more, I left the kitchen. In passing the door of that sanctum some time
after, I caught the words --
   'She'll happen do better for him nor ony o' t' grand ladies.' And
again, 'If she ben't one o' th' handsomest, she's noan faa\l, and varry
good-natured; and i' his een she's fair beautiful, onybody may see
that.'
   I wrote to Moor House and to Cambridge immediately, to say what
I had done: fully explaining also why I had thus acted. Diana and

                            474

                 JANE EYRE                      475

Mary approved the step unreservedly. Diana announced that she
would just give me time to get over the honeymoon, and then she
would come and see me.
   'She had better not wait till then, Jane,' said Mr Rochester, when I
read her letter to him; 'if she does, she will be too late, for our honey-
moon will shine our life long: its beams will only fade over your
grave or mine.'
   How St John received the news I don't know: he never answered
the letter in which I communicated it: yet six months after he wrote
to me, without, however, mentioning Mr Rochester's name or allud-
ing to my marriage. His letter was then calm, and though very serious,
kind. He has maintained a regular, though not very frequent correspond-
ence ever since: he hopes I am happy, and trusts I am not of those who
live without God in the world, and only mind earthly things.

      

This transcription suffers from a number of shortcomings:

• the page numbers and running titles are intermingled with the text in a way which makes it difficult for software to disentangle them;
• no distinction is made between single quotation marks and apostrophe, so it is difficult to know exactly which passages are in direct speech;
• the preservation of the copy text's hyphenation means that simple-minded search programs will not find the broken words;
• the accented letter in faàl and the long dash have been rendered by ad hoc keying conventions which follow no standard pattern and will be processed correctly only if the transcriber remembers to mention them in the documentation;
• paragraph divisions are marked only by the use of white space, and hard carriage returns have been introduced at the end of each line. Consequently, if the size of type used to print the text changes, reformatting will be problematic.

This particular encoding represents a set of choices or priorities. As a trivial example, note that in the second example, end-of-line hyphenation has been silently removed. Conceivably Brontë (or her printer) intended the word ‘honeymoon’ to appear as ‘honey-moon’ on its second appearance, though this seems unlikely: our decision to focus on Brontë's text, rather than on the printing of it in this particular edition, makes it impossible to be certain. This is an instance of the fundamental selectivity of any encoding. An encoding makes explicit only those textual features of importance to the encoder. It is not difficult to think of ways in which the encoding of even this short passage might readily be extended. For example:

• a regularized form of the passages in dialect could be provided;
• footnotes glossing or commenting on any passage could be added;
• pointers linking parts of this text to others could be added;
• proper names of various kinds could be distinguished from the surrounding text;
• names could be classified as personal, geographical, or institutional
• detailed bibliographic information about the text's provenance and context could be prefixed to it;
• a linguistic analysis of the passage into sentences, clauses, words, etc., could be provided, each unit being associated with appropriate category codes;
• the text could be segmented into narrative or discourse units;
• systematic analysis or interpretation of the text could be included in the encoding, with potentially complex alignment or linkage between the text and the analysis, or between the text and one or more translations of it;
• passages in the text could be linked to images or sound held on other media.

TEI-recommended ways of carrying out most of these are described in the remainder of this document. Consult the full Guidelines for more detail.

3.2. The Structure of a TEI Text

Almost all TEI-conformant texts contain (a) a TEI header (marked up as a <teiHeader> element) and (b) the transcription of the text proper (marked up as a <text> element). These two elements are combined together to form a single <TEI> element, which must be declared within the TEI namespace. The full name of the <TEI> element is < xmlns="http://www.tei-c.org/ns/1.0"> .

The header is described in more detail in section 3.17. The Electronic Title Page. The word "almost" in the paragraph above acknowledges the existence of a form of TEI encoding that seeks to capture what is often called the materiality of the text. Some of the relevant element names , <facsimile>, <surface>, <zone>, give you a sense of this approach, where the content of element would be images or other media types rather than transcribed text. While this set of elements is outside the scope of TEI Simple, digital representations that combine transcription with facsimiles are likely to be more common in the future. The global facs attribute is the obvious device for linking pages or other chunks of transcribed TEI Simple text to facsimiles. For more information consult the section on Digital Facsimiles in the Guidelines.

A TEI text may be unitary (a single work) or composite (a collection of single works, such as an anthology). In either case, the text may have an optional front or back. In between is the body of the text, which, in the case of a composite text, may consist of groups, each containing more groups or texts.

Note that in the example above the text between "<!--" and "-->" is a comment that is ignored by the XML parser. You can use the same convention for interspersing comments in your XML document, and it will be similarly ignored. Why then does it show up in the example? Everything in the example is wrapped in a special <egXML> element, which lets you display "raw" XML fragments without the XML parser acting on them.

In the remainder of this document, we discuss chiefly simple text structures, but composite texts are briefly discussed in 3.15. In each of the following sections we begin with a short list of relevant TEI elements and give their formal definition, followed by definitions for any attributes specific to that element. These references are linked to full specifications for each object, as given in the TEI Guidelines. In most cases, short examples are also given.

For example, here are the elements discussed so far:

• TEI (TEI document) contains a single TEI-conformant document, containing a single TEI header, a single text, one or more members of the model.resourceLike class, or a combination of these. A series of <TEI> elements may be combined together to form a <teiCorpus> element.
• teiHeader (TEI header) supplies the descriptive and declarative information making up an electronic title page for every TEI-conformant document.
• text contains a single text of any kind, whether unitary or composite, for example a poem or drama, a collection of essays, a novel, a dictionary, or a corpus sample.
• teiCorpus contains the whole of a TEI encoded corpus, comprising a single corpus header and one or more TEI elements, each containing a single text header and a text.

3.3. Encoding the Body

As indicated above, a simple TEI document at the textual level consists of the following elements:

• front (front matter) contains any prefatory matter (headers, title page, prefaces, dedications, etc.) found at the start of a document, before the main body.
• group contains the body of a composite text, grouping together a sequence of distinct texts (or groups of such texts) which are regarded as a unit for some purpose, for example the collected works of an author, a sequence of prose essays, etc.
• body (text body) contains the whole body of a single unitary text, excluding any front or back matter.
• back (back matter) contains any appendixes, etc. following the main part of a text.

Elements specific to front and back matter are described below in section 3.16. Front and Back Matter. In this section we discuss the elements making up the body of a text.

3.4. Page and Line Numbers

Page and line breaks etc. may be marked with the following elements.

• pb/ (page break) marks the start of a new page in a paginated document.
• lb/ (line break) marks the start of a new (typographic) line in some edition or version of a text.
• milestone/ marks a boundary point separating any kind of section of a text, typically but not necessarily indicating a point at which some part of a standard reference system changes, where the change is not represented by a structural element.

The <pb/> and <lb/> elements are special cases of the general class of milestone elements which mark reference points within a text. The generic <milestone/> element can mark any kind of reference point: for example, a column break, the start of a new kind of section not otherwise tagged, or in general any significant change in the text not marked by an XML element. Milestone elements have no content; whatever they have to say is contained in their attributes, most frequently a <unit> attribute in combination with an n attribute. Their "empty" status is marked by the fact that their opening and closing tags are collapsed into one: the notation <milestone/> is a shortcut for <milestone></milestone>.

Line breaks in prose are for the most part typographical accidents, and many encoders will rightly think of them as not worth their attention. But sometimes you do want to make the transcription mirror the physical layout of the written word on the page. This is especially true of manuscripts where the precise alignment of image and transcription makes things easier for the reader.

Digital transcriptions produced with optical character recognition (OCR) create special opportunities and challenges. They mimic the boundaries of characters, words, and lines on the page, and from their "whitespace XML" you can algorithmically infer much structural encoding, although the algorithmic constructs will always need some manual tweaking. OCR texts also require careful proofreading. This work will clearly benefit from the line-by-line correspondence of page image and transcription.

End-of-line (EOL) hyphens are a vexing problem for which there is no good solution. For many purposes it may be good enough to get rid of them tacitly. Given

                                          I am not per-
               suaded that EOL hyphens are worth keeping.
           

you can turn this into

I am not persuaded that EOL hyphens are worth keeping.

If you want to mark the line breaks of the printed source you could remove the EOL hyphen and keep the word on the line where it began, as in

           I am not persuaded 
           <lb/> that EOL hyphens are worth keeping

This is a perfectly acceptable solution, and few readers will object to it, especially if you document your practice in the relevant section of the header. The downside of both solutions is that you may accidentally delete what you think is an EOL hyphen when if fact it is a real hyphen. There are ways of protecting yourself against such an error. Check if that word occurs with a hyphen in line-medial position. If it does not, you are probably safe. In the unlikely event that it occurs in line-medial position both with and without a hyphen you would not know which it is in the line-terminal position. But relax: there is textual authority for whichever way you jump.

3.5. Marking Highlighted Phrases

3.6. Notes

A note is any additional comment found in a text, marked in some way as being out of the main textual stream. It is a special and very common form of linking, because a note always targets some part of the text, whether implicitly or explicitly. All notes should be marked using the same tag, note, whether they appear as block notes in the main text area, at the foot of the page, at the end of the chapter or volume, in the margin, or in some other place:

• note contains a note or annotation.

Notes may be in a different hand or typeface, may be authorial or editorial, and may have been added later. Use type and resp attributes to distinguish between different kinds of notes or identify their authors.

A note is typically linked to a point of attachment that is represented by a siglum such as an alphanumerical or other character, typically in superscripted form. When encoding such a text, it is conventional to replace this siglum by the content of the annotation, duly marked up with a note element. This may not always be possible for example with marginal notes, which may not be anchored to an exact location. For ease of processing, it may be adequate to position marginal notes before the relevant paragraph or other element.

If the note lacks an explicit point of attachment in the text, which is often the case with marginal notes, you can use the pointer element <ptr>, which differs from <ref> in being empty.

3.7. Cross References and Links

Explicit cross references or links from one point in a text to another in the same or another document may be encoded using the elements described in this section. Implicit links (such as the association between two parallel texts, or that between a text and its interpretation) may be encoded using the linking attributes discussed in section 3.7.3. Special kinds of Linking.

3.8. Editorial Interventions

The process of encoding an electronic text has much in common with the process of editing a manuscript or other text for printed publication. In either case a conscientious editor may wish to record both the original state of the source and any editorial correction or other change made in it. The elements discussed in this and the next section provide some facilities for meeting these needs.

... for his nose was as sharp as a pen and a table of green feelds

3.9. Names, Dates, and Numbers

The TEI scheme defines elements for a large number of ‘data-like’ features which may appear almost anywhere within almost any kind of text. These features may be of particular interest in a range of disciplines; they all relate to objects external to the text itself, such as the names of persons and places, numbers and dates. They also pose particular problems for many natural language processing (NLP) applications because of the variety of ways in which they may be presented within a text. The elements described here, by making such features explicit, reduce the complexity of processing texts containing them.

3.10. Lists

The element <list> is used to mark any kind of list. A list is a sequence of text items, which may be numbered, bulleted, or arranged as a glossary list. Each item may be preceded by an item label (in a glossary list, this label is the term being defined):

• list contains any sequence of items organized as a list.
• item contains one component of a list.
• label contains any label or heading used to identify part of a text, typically but not exclusively in a list or glossary.

Where the internal structure of a list item is more complex, it may be preferable to regard the list as a table, for which special-purpose tagging is defined below (3.12. Tables).

Lists of bibliographic items should be tagged using the <listBibl> element, described in the next section.

3.11. Bibliographic Citations

It is often useful to distinguish bibliographic citations where they occur within texts being transcribed for research, if only so that they will be properly formatted when the text is printed out. The element <bibl> is provided for this purpose. Where the components of a bibliographic reference are to be distinguished, the following elements may be used as appropriate. It is generally useful to mark at least those parts (such as the titles of articles, books, and journals) which will need special formatting. The other elements are provided for cases where particular interest attaches to such details.

• bibl (bibliographic citation) contains a loosely-structured bibliographic citation of which the sub-components may or may not be explicitly tagged.
• author in a bibliographic reference, contains the name(s) of an author, personal or corporate, of a work; for example in the same form as that provided by a recognized bibliographic name authority.
• biblScope (scope of bibliographic reference) defines the scope of a bibliographic reference, for example as a list of page numbers, or a named subdivision of a larger work.
• date contains a date in any format.
• editor contains a secondary statement of responsibility for a bibliographic item, for example the name of an individual, institution or organization, (or of several such) acting as editor, compiler, translator, etc.
• publisher provides the name of the organization responsible for the publication or distribution of a bibliographic item.
• pubPlace (publication place) contains the name of the place where a bibliographic item was published.
• title contains a title for any kind of work.

For lists of bibliographic citations, the <listBibl> element should be used; it may contain a series of <bibl> elements.

3.12. Tables

Tables represent a challenge for any text processing system, but simple tables, at least, appear in so many texts that even in the simplified TEI tag set presented here, markup for tables is necessary. The following elements are provided for this purpose:

• table contains text displayed in tabular form, in rows and columns.
• row contains one row of a table.
• cell contains one cell of a table.

3.13. Figures and Graphics

Not all the components of a document are necessarily textual. The most straightforward text will often contain diagrams or illustrations, to say nothing of documents in which image and text are inextricably intertwined, or electronic resources in which the two are complementary.

The encoder may simply record the presence of a graphic within the text, possibly with a brief description of its content, and may also provide a link to a digitized version of the graphic, using the following elements:

• graphic indicates the location of an inline graphic, illustration, or figure.
• figure groups elements representing or containing graphic information such as an illustration, formula, or figure.
• figDesc (description of figure) contains a brief prose description of the appearance or content of a graphic figure, for use when documenting an image without displaying it.

Any textual information accompanying the graphic, such as a heading and/or caption, may be included within the <figure> element itself, in a <head> and one or more <p> elements, as also may any text appearing within the graphic itself. It is strongly recommended that a prose description of the image be supplied, as the content of a <figDesc> element, for the use of applications which are not able to render the graphic, and to render the document accessible to vision-impaired readers. (Such text is not normally considered part of the document proper.)

3.14. Analysis

3.15.

3.16. Front and Back Matter

3.17. The Electronic Title Page

Every TEI text has a header which provides information analogous to that provided by the title page of printed text. The header is introduced by the element <teiHeader> and has four major parts:

• fileDesc (file description) contains a full bibliographic description of an electronic file.
• encodingDesc (encoding description) documents the relationship between an electronic text and the source or sources from which it was derived.
• profileDesc (text-profile description) provides a detailed description of non-bibliographic aspects of a text, specifically the languages and sublanguages used, the situation in which it was produced, the participants and their setting.
• revisionDesc (revision description) summarizes the revision history for a file.

A corpus or collection of texts with many shared characteristics may have one header for the corpus and individual headers for each component of the corpus. In this case the type attribute indicates the type of header. <teiHeader type="corpus"> introduces the header for corpus-level information.

Some of the header elements contain running prose which consists of one or more <p>s. Others are grouped:

• Elements whose names end in Stmt (for statement) usually enclose a group of elements recording some structured information.
• Elements whose names end in Decl (for declaration) enclose information about specific encoding practices.
• Elements whose names end in Desc (for description) contain a prose description.

[title of source]: a machine readable transcription [title of source]: electronic edition A machine readable version of: [title of source]

4.1. The <model> element with some of its attributes and children

4.2. More about Behaviours

The behaviour attribute has 26 different values, each of them describing what needs to be done to a TEI element for it to be transformed into part of a Web document, PDF file,or other output. We list them below, together with the names of the elements to which they apply, in descending order of the number of elements with which they are associated:

• INLINE (49) creates inline element out of content if there's something in <outputRendition>, use that formatting; otherwise just show text of selected content.: abbr, actor, add, am author, bibl, c, choice, corr, date, del, desc, docAuthor, docDate, docEdition, docImprint, ex, expan, figDesc, figure, foreign, formula, formula, gap, handShift, hi, label, measure, milestone, name, note, num, orig, pc, ref, reg, rhyme, rs, s, salute, seg, sic, signed, space, subst, supplied, time, unclear, w, quote
• BLOCK (35) create a block structure: address, addrLine, argument, back, body, byline, closer, dateline, div, doctitle, epigraph, figure, floatingText, formula, front, fw, group, head, imprimatur, l, lg, listBibl, note, opener, postscript, q, quote, role, roleDesc, salute, signed, sp, speaker, titlePage, titlePart, trailer
• OMIT (10) do nothing, do not process children: author, docAuthor, docDate, docEdition, docImprint, docTitle, encodingDesc, profileDesc, publisher, pubPlace, revisionDesc
• LIST (4) create a list by following content: castGroup, castList, list, listBibl
• BREAK (3) create a line, column, or page break according to type: cb, lb, pb
• LISTITEM (3) create list item: bibl, castItem, item
• ALTERNATE (2) : create a specialized display of alternating elements for displaying the preferred version and an alternative, both at once or by some method of toggling between the two: date,choice
• PARAGRAPH (2) create a paragraph out of content.: ab, p
• ANCHOR create an anchor with ID: anchor
• BODY create the body of a document body:
• CELL create a table cell: cit
• CIT show the content, with an indication of the source: cit
• DOCUMENT start a new output document: TEI
• FIGURE make a figure with the title as caption: figure
• GLYPH show a character by looking up reference to a chardesc at the given URI: g
• GRAPHIC if url is present, uses it to display graphic, else display a placeholder image. heading: graphic
• HEADING creates a heading.: head
• INDEX generate list according to type: body
• LINKcreate hyperlink: ref
• METADATA create metadata section: teiHeader
• NOTE create an note, often out of line, depending on the value of place; could be margin, footnote, endnote, inline: note
• ROW create a table row: row
• SECTION create a new section of the output document: div
• TABLE create a table: table
• TEXT create literal text: title
• TITLE create document title: fileDesc

There are several things to be said about this list. First, the names of the behaviours as well as their explanations, will be familiar to programmers who work with texts on the Web. Secondly, there are far fewer behaviours (26) than elements.

A few processing instructions are quite complex. Take the behaviour alternate, which allows you to toggle between two different elements that are alternative child elements of <choice> and lets you choose between

• an abbreviation and its expansion (<abbr> ad <expan>
• an error in the text and its correction (<sic> ad <corr>)
• an orthographic variant and its regular form (<orig> ad <reg>)

4.3. Customizing the Processing Model

6.1. Elements

data.version

element TEI
{
   att.global.attributes,
   attribute version { data.version }?,
   ( teiHeader, ( ( model.resourceLike+, text? ) | text ) )
}

element ab
{
   att.global.attributes,
   att.typed.attributes,
   att.declaring.attributes,
   att.fragmentable.attributes,
   macro.paraContent
}

data.enumerated

element abbr
{
   att.global.attributes,
   att.source.attributes,
   att.typed.attribute.subtype,
   attribute type { data.enumerated }?,
   macro.phraseSeq
}

element abstract
{
   att.global.attributes,
   att.source.attributes,
   ( model.pLike | model.listLike )+
}

element actor { att.global.attributes, macro.phraseSeq }

element add
{
   att.global.attributes,
   att.transcriptional.attributes,
   att.placement.attributes,
   att.typed.attributes,
   macro.paraContent
}

element addSpan
{
   att.global.attributes,
   att.transcriptional.attributes,
   att.placement.attributes,
   att.typed.attributes,
   att.spanning.attributes,
   empty
}

element addrLine { att.global.attributes, macro.phraseSeq }

element address
{
   att.global.attributes,
   ( model.global*, ( ( model.addrPart ), model.global* )+ )
}

element am
{
   att.global.attributes,
   att.typed.attributes,
   att.editLike.attributes,
   ( text | model.gLike | model.pPart.transcriptional )*
}

element anchor { att.global.attributes, att.typed.attributes, empty }

element appInfo { att.global.attributes, model.applicationLike+ }

data.name

data.versionNumber

element application
{
   att.global.attributes,
   att.typed.attributes,
   att.datable.attributes,
   attribute ident { data.name },
   attribute version { data.versionNumber },
   ( model.labelLike+, ( model.ptrLike* | model.pLike* ) )
}

element argument
{
   att.global.attributes,
   ( ( model.global | model.headLike )*, ( ( model.common ), model.global* )+ )
}

data.enumerated

element att
{
   att.global.attributes,
   attribute scheme { data.enumerated }?,
   data.name
}